1. Technical Field
This invention relates to the field of file servers, and in particular to a method and system for improving load balancing in a file server providing sequential file retrieval. More particularly, this invention relates to reducing disk bottlenecks due to fluctuating user demands generated for the continuous delivery in real time of large files including multimedia files, such as video movies.
2. Background Information
The concept of using file servers to allow users access to a large variety of data objects, including movies, has grown significantly over the last several years, with indications that such use will continue through the next decade. Fueling user demand is the potential ease of access to thousands upon thousands of different files, such as movie files.
In file servers, including large multimedia file servers, access to files, including video movie files, on storage devices, such as disks, may be limited by the number of access or retrieval streams capable of accessing the files on the storage devices. When a large number of users demand access to files on a disk, their access may require the use of all available retrieval streams to the disk, and a condition known as disk bottlenecking will occur. Essentially, with a bottlenecked disk, no bandwidth is available to service any further user requests from that disk. Disk bottlenecking, or as it is sometimes referred, load imbalancing, typically occurs when user demand for access to files on a disk is greater than the number of retrieval streams capable of accessing the files. When disk bottlenecking occurs a particular number of users desiring access to the files on the disk are incapable of accessing the desired files due to the unavailability of retrieval streams.
Avoiding such disk bottlenecks due to the imbalance in resource utilization in a multimedia server is critical because of the stringent response time requirements of sequentially based multimedia file server systems.
It is therefore desirable to reduce disk bottlenecks and load imbalances on large multimedia file servers so that greater user demand can be satisfied.
Disk bottlenecking or load imbalancing can be prevented by spreading each file over all the disks in the system. However, spreading each file over all the disks in the system will result in a time-consuming reconfiguration process when a new disk is added to or dropped (due to failure) from the system.
An alternative way to avoid disk bottlenecking is a static approach where all or some of the files in the system are replicated, in their entirety, onto different disks in the system. However, replicating some of the files requires advanced notice of the number of users desiring access to those files. If advanced notice of the number of users desiring access to certain files is known, then those "hot", i.e. highly accessed, files may be replicated prior to retrieval by the users, thereby avoiding a disk bottleneck.
The problem, however, is that the number of users desiring access to certain files at a particular time is not always known and therefore obtaining advance notice of the same is not always possible. Therefore, it is very difficult to respond to user demand fluctuations.
Also, since disk replication space may be limited there may not be sufficient storage area for the replication of complete files on other disks. Furthermore, such space considerations are even more critical when one desires the replication of all files onto all disks. Finally, since the static replication of all the files over all the disks in the system requires considerable disk space, considerable time and cost also will be incurred.
These problems have prevented an adequate system for avoiding load imbalances in file servers, particularly multimedia file servers, which are used by end users to access movie files.
It is therefore desirable to provide an improved load balancing system.
It is also desirable to provide a method and system for improving load balancing in a file server capable of operating without advanced knowledge of the number of users desiring access to a file at a particular time.
It is also desirable to provide a method and system for improving load balancing in a file server where user demand fluctuates.
It is also desirable to provide a method and system for improving load balancing in a file server which is capable of detecting an overload condition while a file is being accessed by one or more retrieval streams.
It is also desirable to provide a method and system for improving load balancing in a file server which is capable of dynamically avoiding load imbalances as users attempt to access a particular file.
It is further desirable to provide a method and system for improving load balancing in a file server which is capable of replicating portions of accessed files onto other storage devices.
It is further desirable to provide a method and system for improving load balancing in a file server which is capable of permitting files to be read from a plurality of disks.
It is also desirable to provide a method and system for improving load balancing for the continuous delivery in real time of large multimedia files such as movies.